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THE GRADIENT FLOW OF THE POLARIZATION MEASURE. 

WITH AN APPENDIX. 

GIOVANNI PISTONE AND MARIA PIERA ROGANTIN 


Abstract. The polarization measure is the probability that among 3 individuals chosen 
at random from a finite population exactly 2 come from the same class. This index is 
maximum at the midpoints of the edges of the probability simplex. We compute the 
gradient flow of this index that is the differential equation whose solutions are the curves 
of steepest ascent. Tools from Information Geometry are extensively used. In a time 
series, a comparison of the estimated velocity of variation with the direction of the 
gradient field should be a better index than the simple variation of the index. 
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1. Introduction 

Given a discrete distribution vr on n -|- 1 classes x = 0, 1, ..., n, we consider an index 
called polarization measure, defined by 

n 

(1) POL(7r) = ^7r2(l-7r^). 

a ;=0 
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Figure 1. Normalised Polarization. The display shows: the proba¬ 
bility simplex as an equilateral triangle; the level curves of 4 x POL; 
the unstable critical point at vr = (1/3.1/3,1/3) (circle); the mini¬ 
mum points at the vertexes (triangles); the maximum points at vr = 
(0,1/2,1/2), (1/2, 0,1/2), (1/2,1/2, 0) (squares). 


The polarization measure has been introduced in Economics for real distributions by 


Esteban and Ray 

1994 

and Vidal-Robert 

2013 


P- 


The discrete version we consider here has been used in Pino 

10 ], 

The polarization measure has the following interpretation. Let X, Y, Z be i.i.d. ~ n 
and consider the indicator of exactly two equal 

I^ = {X = Y ^ Z) + {X = Z ^Y) + {Y = Z ^X). 

Then E [h] = 3 ELo ^"(1 - vr.) = 3 POL (vr). 

The polarization measure on the classes {0,1, 2}, as shown in Fig. has an unstable 
critical point at the uniform distribution, it is zero in the case of concentration in one class, 
and reaches its maximum 1/4 on distributions on two classes with equal probabilities. 
Polarization measure was devised to be an index of the distance of a distribution from 
the three cases of maximal polarization. In Fig. the simplex is represented as an 
equilateral triangle. In the following we shall use different sets of coordinates to represent 
the probability simplex, e.g. see Fig. I (left). 

We want to study the dynamics of this index, i.e. to characterise evolutions that 
maximise or minimise the index. This study requires tools from Information Geometry 


(IG) e.g., Amari and Nagaoka 2000 , Gibilisco and Pistone 1998 , Pistone and Rogantin 
|1999| , Gibilisco et ah 


2010 , Pistone 2013 , Malagb and Pistone 


2014 . However, the 


following presentation is actually largely self-contained. 

The recourse to IG is not dispensable because the ordinary gradient flow of POL, as 
shown in Fig. (left), does not lead to the extrema of interest on the border of the 
probability simplex. Gonsequently, one wants to turn to a different way to compute the 
gradient, i.e. to the so-called Amari’s natural gradient. We use elementary fact of the 
theory of Dynamical Systems to characterise critical points of the gradient flow and refer 
to Abraham et ah 1988 . 

The basics of IG are discussed in Sec. The application of IG to the polarization 
measure is described in Sec. The possibility of a generalisation of such an index 
is shortly discussed in Sec. while the reduction of the problem to the study of an 
exponential family is presented in Sec. We suggest a possible application in Sec. 

Further material, not directly related with the measure of polarization, but suggested 
by the methodology, is presented in the Appendixes. Differential equations on the proba¬ 
bility simplex are well known in applications other then Descriptive Statistics. We briefly 
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discuss the relations between these applications and our one in App. [At In App. some 
issues related to the second order calculus are briefly discussed. 


2. Natural gradient 

We denote by A„ the simplex of the probability function tt on 0,1, • • •, R- The interior 
of the simplex, A°, is the set of the strictly positive probability functions, 

n 

TTa, = 1, TTg; > 0, x = 0,1,..., ra 

1=0 

The border of the simplex is the union of all the faces of A„ as a convex set. We recall 
that a face of maximal dimension n — 1 is called facet. A facet is a simplex of dimension 
n — 1. 

We define to be the vector space of random variables U that are 7r-centered, [f/] = 
0. In the geometry of Bt^ is the plane through the origin, orthogonal to the vector 


A° = <^ TT e 


pn+l 



Definition 1. 

(1) The tangent bundle of the open simplex A° is the set 

TAl = {{7r,U)\neAl,U eB,}. 

(2) If I 3 t ^ pit) E A° is a one-dimensional statistical model, geometrically a curve, 
its score 

= i) = I 

belongs to i?p(t) for all t E I. As the score is a centered random variable, hence 
I 3 t e3- {p{t), Dp{t)) is a curve in the tangent bundle. 

In fact, U E i? 7 r is meant to represent a generic velocity vector through tt, see Fig. 
1^ The score is a representation of the velocity along a curve, because of a geometric 
interpretation of C. R. Rao’s classical computation: 


( 2 ) ^Et[U] = ^^U{x)p{x]t) = J2u{x)^p{x-,t) = 


^U{x)^\og{p{x;t))p{x;t) = ^{U{x) - Et[U]) ^ log {p{x;t)) p{x;t) 


d 


[U-E,[U])-log {p{t)) 


= {U-E,[U],Dp{t))^^,^ 


We observe that the scalar product above is the scalar product on i?p(p. 

A curve on the simplex is a parametric model. The probability vr is represented by 
a vector from O to the point whose coordinates are In Fig. the velocity 

vectors are represented by arrows; they are orthogonal to the vectors On. 

In our context, a vector field E: A° -3 is a mapping such that E^n) E i.e. 
such that the couple (vr, E^n)) belongs to the tangent bundle, (vr, E^n)) E TA° for all vr. 
Because of our geometrical construction, here we prefer to call E a vector held, but we 
want to stress its statistical meaning of centered function of the distribution. 

A differential equation is an equation of the form Dpif) = E{p{t)). 
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Figure 2. The simplex (solid triangle) is view from below. The curve on 
the simplex is a parametric model. The probabilities tt are represented by 
vectors from O to the point whose coordinates are vr. The velocity vectors 
are represented by arrows; they are orthogonal to the vectors from O to vr. 


Given a real function 0: A° —>■ M, its gradient is the vector held V0 such that for all 
curves p(-) we have 

(3) ^ (V0(p(t)), , (u, v)^ = [uv ], M, n e B^. 

The Rao’s computation in Eq. ([^ is the prototypical gradient computation. 

The gradient flow eguation is the differential equation 


Dpfl) = V(/)(p(f)). 


Along a solution of the gradient how equation the value of 0 is increasing because 
d(j){p(t))/dt = (V0(p(f)), > 0. Actually the solution of the gradient how 

equation is the curve of steepest ascent. 

Computations are usually performed in a parametrization 

tt: 0 3 0 I-)- 7r(0) G A°, 


0 being an open set in M”. The j-th coordinate curve is obtained by hxing the other 
n — 1 components and moving 9j only. The scores of the j-th coordinate curves are the 
random variables 

^i7r(0) = ^log7r(0), j = 

The sequence {Dj7i{0): j = 1,... ,n) is a vector basis of the tangent space R 7 r( 0 )- The 
representation of the scalar product in such a basis is 


t = l / 7r(0) 


. *=1 


i,i=l 


^ ^ ( 0 ), 

*J=1 

is the Fisher information matrix. 


where the matrix /(0) = {Di7r{6), Dj7r{6))^f^g.^ 

If 0 H->■ 0(0) is the expression in the parameters of a function 0: A° —)■ M, that is 
0(0) = 0(7r(0)), and t h->■ 0(f) is the expression in the parameters of a generic curve 
p: I ^ A°, then the components of the gradient in (|^ are expressed in terms of the 
ordinary gradient by observing that 


7 7 ^ f) 

jmt)) = jflm)) = E 

j=i :! 
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VPOL /■"= VPOL 

I Fisher information 
matrix 


V POL 
gradient 


TK 

tangent bundle 


tangent bundle T ,V P0L(71)) 

tangent bundle jj^n -idu^ /q P? (r\\\ 
coordinates R 5(0,VPOL(0)) 


Figure 3. Diagram of the action of the natural gradient in a given 
parametrization vr: 0 —A°. 


As Dp{t) = we obtain from (|^ 

(4) = (W(pW), (V(j){p{t)), Dj7r{e{t))ej{t)'^. 


). The natural gradient is a vector V(j){6) whose components 
are the coordinates of the gradient V0(7r(0)) G -B,r(6») in hs vr-basis, that is 


Definition 2 


(Amari| 


1998 


v^(.7,{e)) = 5^(v.^(e))pj7r(0). 

i=i 

By substitution of the expression in Q we obtain 


(5) 


V0(6>) = V0(6>)J-^(6>). 


Fig. I is an illustration for the function ^(vr) = POL(7r). 

The common parametrization of the (flat) simplex A° is the projection on the solid 


simplex r„ = jr/ G M" 0 < Vj < ij) fhnt is 


tt: Tn 3 ri 


j=i 


.Vn 


G A 


O 

n") 


in which case dj7T{r]), j = 1,... ,n, is the random variable with values —1 at x = 0, 1 at 
X = j, 0 otherwise, hence dj7i{r]) = ((A = j) — (A = 0)) and 

DMv) = {{X = j) - (A = 0)) /7r(r/). 

The element (j, h) of the Fisher information matrix is 


ijhi-n) 


~ (A=j)-(A = 0) (A = h)-(A = 0) ' ^ 

'"'U 7r(A;r7) 7r{X]r]) 

^7r(x,r/)“^ ((x = j)(j = h) + (x = 0)) = r]-\j = h) + ( 1 - 

X \ k 


I{ri) = diag(r/) ^ + 




[ 1 ] 


n 

*j=i' 


hence 


-1 















As an example we consider n = 3. The Fisher information matrix, its inverse and the 
determinant of the inverse are, respectively. 


HVi,V2,V3) = 


{l-r]i-T]2- r]3) ^ 


hi (1-^2-hs) 


1 

h2^(l - 

1 


(1 -hl)hl 

-V 1 V 2 

-V 1 V 2 

(1 - V2)V. 

-V 1 V 3 

-V 2 V 3 


hi - hs) 1 

L ri^\l - r]i - r]2) 

-hihs 

-h2h3 

(1 - h3)h3 


det {I{rii,ri 2 ,r] 3 ) = (1 - hi - h 2 - h3)hih2h3- 

Note that the compntation of the inverse of I{r]) is an application of the Sherman- 
Morrison formnla and the computation of the determinant of /(r/)“^ is an application of 
the matrix determinant lemma. 

For general n, we have the following Proposition, whose interest stems from the deh- 
nition of natural gradient, see Eq. (|^. 


Proposition 1. 

(1) The inverse of the Fisher information matrix is 

I (r/)“^ = diag (r/) — r]rf 

(2) In particular, is zero on the vertexes of the simplex, only. 

(3) The determinant of the Fisher information matrix is 

( n \ n 

1 - 5^hi) n^*- 

i=i ) i=i 

(4) The determinant of is zero on the borders of the simplex, only. 

(5) On the interior of each facet, the rank of is n — 1 and the n — 1 liner 

independent column vectors generate the subspace parallel to the facet itself. 


Proof. 

(1) By direct computation, is the identity matrix. 

(2) The diagonal elements of are zero if pj = 1 or pj = 0, for j = 1,... ,n. 

If, for a given j, pj = 1, then the elements of are zero ii ph = 0, h ^ j. 

The remaining case corresponds to pj = 0 for all j. Then = 0 on all the 

vertexes of the simplex. 

(3) It follows from Matrix Determinant Lemma. 

(4) The determinant factors in terms corresponding to the equations of the facets. 

(5) Given i, the conditions pi = 0 and pj 7 ^ 0,1 for all j 7 ^ i, define the interior 

of the facet orthogonal to standard base vector Cj. In this case the i-th row 

and the i-th column of are zero and the complement matrix corresponds 

to the inverse of a Fisher information matrix in dimension n — 1 with non zero 
determinant. It follows that the subspace generated by the columns has dimension 
n — 1 and coincides with the space orthogonal to pt. Consider the facet defined by 
(1 — ^11=1 Vi) = 0) hi 7 ^ 0) 1 ^ given j, the matrix without the j-th 

row and the j-th column has determinant i^j Vi On the 

considered facet this determinant is different to zero and has rank n — 1 

and their columns are orthogonal to the constant vector. □ 
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□ 

An other parametrization is the exponential parametrization based on the exponential 
family with sufficient statistics Xj = (X = j), j = 1,..., n, 


TT: 


3 0 I— )■ exp I djXj — '4){9) 
0=1 


n + 1 


where 


tp{9) = log 


(■*?••■) 


- log (n + 1). 


Some of the properties discussed in Prop. [T] should act ually be discussed under the 
exponential parametrization, see e.g. Malagb and Pistone 2014 , but we do not do that 
here. We will discuss the exponential parametrization below in Sec. to show that 
polarization can be seen as an expectation with respect to an exponential family. 


3. The gradient flow of POL 


We apply now the general theory of the natural gradient to the study of the dynamics 
of the polarization measure. Our goal is to hnd the lines of the steepest ascent of the 
function POL. 

In the common parametrization we have 

/ n \ ^ ^ \ ^ 

POL(r/) = 1 - 


i=i 


vi=i 


i=i 


and, for n = 2, 

POL(r/) = ri 2 Yir]i + m) + “ hi) + “ V 2 ) = 

3(h?h2 + Vivl) - ivl + Til) - 4hih2 + (hi + h2) 

with gradient 

VP0L(r7) = { 6 r]ir ]2 + ^vl - 2hi “ 4h2 + 1, 6i72hi + 3hi - ‘^V 2 - + l) ■ 

The inverse of the Fisher information matrix is 


I ^(hl,h2) = 


(1 - r]i) 7 ]i -7J1P2 

-hlh2 (1 - h2)h2 


The natural gradient is 


(6) VPdL(77l,772) = VPdL(77l,772)/"'(hl,h2) = 

{-9r]lr]2 - 9r]lr]l + 2 t]1 + Ur]lr]2 + ^Vivl “ 3h? - ^ViV2 + Vi, 

-9ViVl - 9hih2 + ^ViV 2 + 14?7i?72 + 2rjl - 5r]iT]2 - ^vl + V 2 ) ■ 
See in Fig.s and |^the gradient helds. 

Some properties of the natural gradient held depend on the inverse of Fisher informa¬ 
tion matrix only, other are specihc properties of the function POL. We note that the 
vector held in ([^ is actually dehned and continuous for all r/ G and coincides with 
the natural gradient in the interior P„ of the solid simplex. By abuse of language, we call 
the extended object with the same name of the probabilistic object. 

The inverse of Fisher information matrix is zero at the points (0,0), (0,1), (1,0), 
Prop. [ 1 } These are among of the hxed points of the gradient how equation. The 
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see 









Figure 4. Level curves of the Polarization in the common parametrization 
(left) and natural gradient held (right). 



Figure 5. The gradient without the correction by gives the wrong 
directions (left), while the natural gradient applies correctly (right). Both 
helds are extended outside the probability simplex. The length of the 
arrows is relative to each display and cannot be compared across displays. 


determinant of the inverse of Fisher information matrix is det — — i 72 )hih 2 - 

As proved in Prop. for the general case, the determinant is zero on the borders of 
simplex. Here the probabilistic model is not dehned, but the continuous extension of 
the gradient how holds. On the facets of the simplex the vector held is parallel to the 
facets itself. On the facets the inverse of Fisher information matrix is one dimensional: 
if ? 7 i = 0 then corresponds to ( 0 , 1 )*, if 172 = 0 then corresponds to ( 1 , 0 )*, and if 
1 — rji — ri 2 = 0 then corresponds to ( 1 , —l)h 

To study the how in the hxed points we consider the sign of the eigenvalues of the 


Jacobian of the natural gradient, calculated in the hxed points, see |Arnold 

'avpdL(T7) 


The Jacobian of VPOL(? 7 ) is J = 


dm 


avpoL(T?) 

dm 


2006 


where 


aVPOL(r7) 

dr]i 


-27rifri2 - + 6r]l + 28riir]2 + 7)T]1 


-Qrjl 


+ 107721 ?! 


6771 -5772 + 1 
- 5772 


aVPOL(r/) 

dri2 


- 9 t ]1 - 1877^772 + 1477^ + IO771772 - 5771 
- 27 r]^rji - 1877277^ + 677I + 28772771 + b'qj - 6772 -6771 + 1 
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The Jacobian calculated in the vertexes are 


J(0,0) = J(0,1) = J(1,0) 


1 0 
0 1 


and the two eigenvalues of the three Jacobian are both positive. Then, the fixed points 
on the vertexes repel flow locally. 

Moreover, the natural gradient is zero on the midpoints of the borders, (O, |), (|,0) 
and (|, !)• The Jacobian calculated in the midpoints are 



1 [2 o' 

8 1 4 



1 [4 1 
8 0 2 



1 r 3 -1 

8-13 


and the two eigenvalues of the three Jacobian are both negative. Then, these hxed points 
attract flow locally. 

In standard cases the Jacobian of a gradient is the Hessian matrix. This is not true 
anymore in IG where we compute the Jacobian of the natural gradient. Actually, in IG 
there are various notions of Hessian, each one based on a different connection on the 
tangent bundle. 


4. Generalisation of the polarization measure 


We consider now an inverse problem in the simple case n = 2. We want a third degree 
symmetric polynomial which could be used as polarization measure. Gomputations were 


performed using Sage Stein et al. 2014 


A generic polynomial is 


/(tt) = a(7r^ + TT? + vr^) + b(7r^7ri + ttoTT^ + + vtoTT^ + vr^TTa + 7ri7r^) + 

CTToTTiTTa + (J(7rg + TT^ + TT^) + e(7ro7ri + TToTTa + TTiTTa), 

and its expression in the parameters is 

f(rf) = (—3a + 3b — c)rilri 2 + (—3a + 3b — c)riir]l + (3a — b + 2d — e)ril+ 

(6a — Ab + c + 2d — e)rjirj 2 + (3a — b + 2d — e)rjl+ 

(—3a -\~ b — 2d T T (—3a -\~ b — 2d T c)‘i72 T a T d. 

The components of the natural gradient are 
(V /)i(?7) =(9a — 96 + 3c)ri\ri2 + (9a — 96 + 3c)rilr]l + (—6a + 26 — 4d + 2e)T]l+ 
(—18a + 146 — 4c — 4d + 2e)rjlrj2 + (—9a + bb — c — Ad + 2e)riiril+ 

(9a — 36 + 6d — 3e)ril + (9a — 56 + c + 4d — 2e)rjirj2+ 

(—3a + 6 — 2d + e)rji 

(V f) 2 ('n) =(9fl — 96 + 3c)t]iT] 2 + (9a — 96 + 3c)r]lril + (—6a + 26 — 4d + 2e)r]2 + 
(—18a + 146 — 4c — 4d + 2e)riiril(—9a + 56 — c — 4d + 2e)rjlr]2+ 

(9a — 36 + 6d — 3e)rj2 + (9a — 56 + c + 4d — 2e)r]irj2+ 

(—3a + 6 — 2d + e)ri 2 . 

Note that the case / = POL is 6 = 1 and a = c = d = e = 0. Because of the 
multiplication by I(r])~^, the natural gradient V/(r/) is zero in each of the simplex 
vertexes (0,0), (1,0), (0,1). Because of the symmetry, the natural gradient V/(r/) is 
zero in each of the mid-point of the edges, i.e. (1/2,0), (0,1/2), (1/2,1/2) and at the 
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uniform probability, (1/3,1/3). We need to turn to the discussion of the Jacobian matrix 
of the natural gradient of /. 

At the vertexes, the Jacobian is 


(—3a + b 


2d + e) 


0 

1 ’ 


so that we want —3a + b — 2d + e > 0. 

At the uniform probability (1/3,1/3) the Jacobian is 


1 

9 


( 6 a 


c + 6 — 3e) 


1 

0 


0 

1 


A necessary condition of non dehniteness is 


(7) 


6 a — c + 6 (J — 3e = 0. 


At the mid-points of the three edges (1/2,0), ( 0 , 1 / 2 ), ( 1 / 2 , 1 / 2 ), the values of the 
Jacobian matrices are, respectively. 


^a — \b + d—\e 
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1 . 


3f + ^-2f 
-|a - - d 


-|a - |c - d -f 
|a — Ifo — Ic + d — le 

|a - |6 -f Ic 
— I® + |c — d J- le 


0 

|a — Ife d — lej 


— 8 ® 8 ^ 8 *" — d + 26 

fa - - 1 - Ic 


and, imposing the necessary condition c = 6 a + 6 d — 3e of Eq. ([^, they are 


1 

8 


(3a — 6 + 2d — e) 


4 1' 

1 . , , ^ 

2 0’ 

1 . , , ^ 

’ 3 -i’ 

0 2 

, -{?>a - b + 2d - e) 

0 

1 4 

, -(3a — 6-|-2d — e) 

8 

-1 3 


The two eigenvalues of each of these matrices are both negative if 3a — 6 -|- 2d — e < 0. 
Notice that this condition of attracting flow on the midpoints is the same condition of 
repelling flow on the vertexes. 

Summarising, the conditions on the coefficients of a third degree symmetric polynomial 
that represent a measure with the properties stated above, are 


6 a — c -|- 6 d — 3e = 0 3a — 6 -|- 2d — e < 0. 


In conclusion, it is possible to design other measures of polarization in the form of 
a symmetric polynomial of degree 3, or more general forms. In particular, it would be 
interesting to have a measure in a form similar to the entropy. The analogy is suggested 
by the fact that the gradient flow of the entropy gives trajectories that move from the 
uniform probability to one of the vertexes of the simplex. 


5. POL AS EXPECTATION ALONG AN EXPONENTIAL FAMILY 

We have already observed that the function POL dehned in Eq. [^is an homogeneous 
polynomial of degree 3 in the indeterminates tTj.. The general class of indexes based on 
homogeneous polynomials that we have discussed in Sec. |^is of special interest because 
they reduce to an expectation with respect to an exponential family, as we discuss now 
in the case of three sample points. 
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Table 1. Sample space and random variables of the exponential family in 
Eq. ([^. The sample cases of polarization are presented in boldface in the 
hrst column. 



A 

Y 



El 


w 

E 2 

^2 

Ti 

T 2 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

2 

1 

0 

0 

1 

0 

0 

0 

0 

0 

1 

0 

3 

2 

0 

0 

0 

0 

0 

1 

0 

0 

0 

1 

4 

0 

1 

0 

0 

1 

0 

0 

0 

0 

1 

0 

5 

1 

1 

0 

1 

1 

0 

0 

0 

0 

2 

0 

6 

2 

1 

0 

0 

1 

0 

1 

0 

0 

1 

1 

7 

0 

2 

0 

0 

0 

0 

0 

1 

0 

0 

1 

8 

1 

2 

0 

1 

0 

0 

0 

1 

0 

1 

1 

9 

2 

2 

0 

0 

0 

0 

1 

1 

0 

0 

2 

10 

0 

0 

1 

0 

0 

1 

0 

0 

0 

1 

0 

11 

1 

0 

1 

1 

0 

1 

0 

0 

0 

2 

0 

12 

2 

0 

1 

0 

0 

1 

1 

0 

0 

1 

1 

13 

0 

1 

1 

0 

1 

1 

0 

0 

0 

2 

0 

14 

1 

1 

1 

1 

1 

1 

0 

0 

0 

3 

0 

15 

2 

1 

1 

0 

1 

1 

1 

0 

0 

2 

1 

16 

0 

2 

1 

0 

0 

1 

0 

1 

0 

1 

1 

17 

1 

2 

1 

1 

0 

1 

0 

1 

0 

2 

1 

18 

2 

2 

1 

0 

0 

1 

1 

1 

0 

1 

2 

19 

0 

0 

2 

0 

0 

0 

0 

0 

1 

0 

1 

20 

1 

0 

2 

1 

0 

0 

0 

0 

1 

1 

1 

21 

2 

0 

2 

0 

0 

0 

1 

0 

1 

0 

2 

22 

0 

1 

2 

0 

1 

0 

0 

0 

1 

1 

1 

23 

1 

1 

2 

1 

1 

0 

0 

0 

1 

2 

1 

24 

2 

1 

2 

0 

1 

0 

1 

0 

1 

1 

2 

25 

0 

2 

2 

0 

0 

0 

0 

1 

1 

0 

2 

26 

1 

2 

2 

1 

0 

0 

0 

1 

1 

1 

2 

27 

2 

2 

2 

0 

0 

0 

1 

1 

1 

0 
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If the random variables X, Y, Z are i.i.d. with the distribution of X supported by 
{0,1, 2}, and coded as an exponential family i.e. 


X 


^2 

0 

0 

0 

1 

1 

0 

2 

0 

1 


TT^ =exp{eiXi+ 62 X 2 -'ilj{ 0 i, 62 )), 
= log (l + e®i + e®2^ , 


then the joint probability function of X, Y, Z is itself an exponential family, 

( 8 ) p{x, y, z; Oi, 62 ) = x,y,z = 0 , 1 , 2 , 

with sufficient statistics Tj{x, y, z) = Xj + yj + Zj, j = 1,2. The model for Ti, T 2 is 

(9) p{ti, t 2 ] 62 ) = ^ 2 ), 

where / is the table count of Ti,T 2 . The random variables of this model are represented 
in Tab. [H 

The marginal polytope, that is the convex set generated by the values of the sufficient 
statistics, is illustrated in Fig. We refer to Brown] |1986| for the relevant theory. 

If / is the indicator function of the set {(0,1), (0, 2), (1, 0), (1, 2), (2, 0), (2,1)} then the 
value of 3 POL, when expressed in the parameters 6, is Eg [/(Ti, T 2 )]. Thus, the problem 
of the maximisation of the polarization measure is rephrased to the problem of Ending the 
maximum of the expected value of a random variable on a given exponential family. This 
approach has been considered in Combinatorial Optimisation, see e.g 
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Malago 2012 















Ti\T2 0 12 3 
0 13 3 1 

1 3 6 3 0 

2 3 3 0 0 

3 10 0 0 


Figure 6. Marginal polytope of the exponential family. The points in 
the diagram are the valnes of the snfficient statistics. Each one of the 
emphasised points correspond to three cases where polarization occnrs, 
for example Ti = 1,T2 = 2 corresponds to the cases 18 : (2,2,1); 24 : 
(2,1,2); 26 : (1,2,2), in Tab. At the right, the table of joint connts of 
Ti, T 2 is displayed. 


Malagb et ah 2011 , Ollivier et ah 2011vl; 2013v2 , Malago et al. 2013 . It has a 


nnmber of issnes. 

First, any convergent evolntion along the exponential family has either a limit internal 
to the exponential family itself of a limit ontsid e the exponential family, snpported by a 


face of the marginal polytope, see Cencov 


Malagb and Pistone 2010 


1982 


Rinaldo et al. 


2009 


Ranh et al. 


2011 


Second, the maximnm of the fnnction can be reached as a limit of expected valnes if, 
and only if, there exists, among the distribntions obtained conditioning the exponential 
family to one face of the marginal polytope, a distribntion snch that the expected valne 
of the fnnction is eqnal to the maximnm of the fnnction itself. We do not enter here in a 


detailed discnssion, see more information in Malagb and Pistone 2010| 2014 
The expectation parameters of the exponential family (|^ 


are 


Ee [Ti] = 3j-i/;(0i,02) = 3 
Ee [T2] = 3j-i/.(0i,02) = 3: 


.01 


1 + e^i + e®2 ’ 

q02 


862 ^'' 1 + e^i + e ®2 

These parameters are related to the rj by 

Ee [I'j] = E 0 [Xj] + Eg \Yj\ + Eg [Zj] = Srjj, j = 1, 2. 
The inverse of the map 


can be compnted explicitly as 


e^^ = 


: 0 T] 




V2 


l-r]i-r]2 l-r]i-r]2 

The cnmnlant fnnction is expressed as a fnnction of rj as 

O 2 ) = 3 log (1 + e®^ + e^2) = log 


(1 - ?7i - 772 )^’ 
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and the exponential family in Eq. ([^ is expressed in the parameter r] as 


( 10 ) t2) = t^) = 

hl'h2"(l - hi - h2)^“*^"*V(^l,^2) =Pr,(tl,t2). 


When the probabilities are expressed in the form of Eq. (|10|), we can actnally compnte 
valnes for border cases 


known. 


Pistone, 2009 . We recover in a different way the resnlt already 


hi =0,?72 = 1/2:: Eq. @ becomes Po.i/ 2 (^i,^ 2 ) = 0*i(l/2)^ *V(^n^ 2 ), which is 
zero bnt for ti = 0. This distribntion is concentrated on {(0,1), (0,1), (0, 2), (0, 3)}, 
with po, 1 / 2 ( 0 , ^ 2 ) = /(0,t2)/8. It follows Eo,i /2 [/(Ti,T 2 )] = 3/4. 

7 ]i = 1/2,7/2 = 0:: Same as the previons case. The distribntion is concentrated where 
h = 0, Pi/ 2 ,o(^i, 0) = /(ti, 0)/8, and Ei/ 2,0 [/(Pi, P 2 )] = 3/4. 
rji = 1/2, 7/2 = 1/2:: In this case the distribntion is (l/2)*i ^ 2 ) with 

snpport on the face where ti + t 2 = 3, probabilities pi/ 2 ,i/ 2 (t, 3— f) = f(t,3 — t)/8, 
and Ei/ 2 ,i /2 [/(Pi,P 2 )] = 3/4. 


We have shown on an example that the maximal valne of the polarization measnre 
is actnally reachable as a limit of the expected valne of a random variable /(Ti,T 2 ) on 
the exponential family, bnt this does not mean that the maximnm valne of the expected 
valne is eqnal to the maximnm valne of the random variable itself. This wonld be trne if 
the maximnm of the random variable were reached on a face of the marginal polytope. 
This is discnssed in the literatnre we have cited above. 


6. Conclusion and suggested applications 


We have considered a statistical index different from indexes for concentration or nni- 
formity, snch as the discrete Gini index or Boltzmann-Gibbs-Shannon entropy. The 
polarization measnre has been discnssed in a dynamic way, by considering its variation 
and compnting the directions of steepest variation. The stndy reqnires tools snitable to 
discnss differential eqnation on a differentiable manifold. 

This methodology snggests to implement the velocity of variation itself as a sta¬ 
tistical index. Gonsider a stndy of the evolntion of an index 

[M3 


Pino 


and Vidal-Robert 


in time snch as 
the evolntion of the index e.g. 


In the time series 771 , 772 , 

POL ( 771 ), POL ( 772 ),..., conld be misleading, becanse an increase in the index conld be 
associated to a shift from a basin of attraction to a different basin of attraction. We 
snggest a more precise local stndy as follows. Given a movement from tti to Tit+i, we look 
for a comparison of an estimate of the velocity vector to the gradient held of the 

index, that is compnte {ntTit+i, V POL (tt*))^^. An estimator of the velocity is a mapping 
from a conple of densities vTinitiai, TT finai to the tangent space at the initial density T,ri„itiai2^n- 
The inverse of snch a mapping is discnssed nnder the name of retraction in Absil et"^ 
The simplest example here being irtTit+i = — 1, which is 


2008 


snggested by 


Dnii) = — log 77 ( 7 ) = lim h 
^ ^ dt ^ ^ h^o 


-1 T^t+h ~ 


It is a common practice in Engineering to nse the initial velocity of the Riemannian 


geodesic connecting tt^ to tt^+i Absil et ah, 2008 . This wonld reqnire the compntation 


of the geodesic itself, which is done nsing the compntations schetched in App. 
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Appendix A. From Lotka-Volterra to the replicator 


A.l. Lotka-Volterra n = 2. From Goel et al. 1971 , with a*. A* > 0, i = 1,2, and 
t HA- {Ni(t), N 2 it)) G M>, the LV equation 

f iVi = oiiNi — X1N1N2 = Ni{cxi — X1N2), 

( 11 ) / . 

[ N2 = —U2N2 + \2NlN2 = N2{ — U2 + A2Y1). 

has a stationary point {qi,q 2 ) with 

( 12 ) 


02 


Ol 


= y-, q2 = . ■ 

A2 Al 


15 







In the variables 


(13) 

the equations are 

(14) 

It follows that 
(16) 

SO that 


iVi AaATi 

= — =-: 

qi 02 

N 2 AiA ^2 

2(2 = = -. 

q2 ai 


zi = ai2;i(l - 2:2), 
is = 022:2 (2:1 - 1 ), 


i.e. 


^ log 2)1 = q;i (1 - Z 2 ), 
^ log 2(2 = 02(^1 - !)■ 


1-Zi . , 1 - ^2 . _ „ 

02- Zi + cxi- Z2 — 0 , 


2^1 


^2 


(16) C'(2 :i, 2:2) = Q;2(log2:i - zi) + ai(logZ2 --22) = log ((zie ^^)°^(z 2 e 

is constant. Note that hni2^^^2^o,+oo <^(^1, ^2) = —00 and 


(17) 


Hess C(2:i, 2:2) = 


^ 0 
0 ^ 


is negative dehnite. 


This shows the existence of periodic orbits, see Goel et ah, 1971, p 10-11]. 

A.2. Uplift of Lotka-Volterra. Because of the periodic behaviour, we cannot expect 
the dynamic project to the simplex Ai. Following Hofbauer 1981|, we can go up to 


(18) 


^2 = {^= (^0,7^1, 7 ^ 2 ) 


Y^TTi = l,7ri > 0,i = 0, 1,2| 


We add a constant population of one individual 2:9 = 1 and dehne the transformation 
^2 o A° 


(19) 


TTo = 
TTi = 
TTs = 


ZO 


Zq + Zi + Z 2 
Zi 


1 + 2(1 + ^2 
2:1 


^0 + + ^2 1 + + ^2 

2^2 _ 2:2 

^0 + + ^2 1 + + 2(2 


2:1 = 


2:2 = 


TTo 

TTo 


Note that '^0 + '^1 + '^2 = -^ (ttq + 7ro2:i + 7ro2:2) = 0. We have 

(20) TTo = ^(1 + 2(1 + Z2)~^ = -vro(ii + is) = -Trl{aiZi{l - Z 2 ) + q;2-22(-2i - 1)) = 


TTo 


2 / / 1 ^2 W TTs / TTi 

- ttq ( ai— I 1-) + 02 — 1-1 


TTo 


TTo V^O 


— — (OlVTi (tTq — TTs) + OsTTs (tTi — TTq)) — 

TTiTTsA 


TTo I —OiTTi + a2'^2 + (c^l — (^ 2 )- 


TTO / ’ 
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/ \ ^ / \ '^1 ( ^2 

(21) ill = = '^0^1 + TToii = TTo-h vToai— 1- 

at TTo TTo \ TTq 


— TTi ( —OfiTTi + a2'^2 + (o^l ~ 0 ^ 2 ) 


7ri7r2 \ 
TTo / 


+ ttiTTi 


1-^ 

TTo 


= TTi 


, x7ri7r2 

-ttiTTi + tt27r2 + (Ol — 0'2) -h Ol 

TTo V ^0 


7^2 

1 - — 


/ \ ^ / \ ^2 ^2 / 

(22) 71-2 = 3T(7ro^2) = T^oZ2 + 71oZ2 = TTo -h 7roQ;2— -1 

at TTq TTq yTTo 


^2 ( —OLiTli + Q!27r2 + (Oi — ^ 2 )- ) + Ot2Tl2 (-1 

V TTq / \7ro 

= ■^2 1 —OtiTli + a27r2 + (oi — 0:2) - \~ 0^2 ( -1 

V ^0 V^o 


Define 

(23) /o(7r) = 0, /i(7r) = ai(l-—\ f 2 A) = «2 (— - 1 

V ^0/ V^o 

Then 


(24) TT ■ /(tt) = 7ro/o(7r) + 7ri/i(7r) + 7i2f2A) = 


TTitti ( 1 - — ] + 77202 ( — - 1 1 = 
TTo/ V^O 


OlTTi — 027I’2 + (Q^2 ~ Oil) 


771772 

TTo 


hence the differential equation for tt is the replicator equation 
(25) 77i = 7ri(/i(7r) - 77 •/(tt)), 7 = 1,2, 3. 

Appendix B. Information Geometry of the replicator in dimension 2 


The replicator equation Eq. (25) is nothing else then a particular class of differential 


equations in A2 as a submanifold of of Let f: 62 ^ where /(tt) G is viewed 
as a random variable on {0,1, 2}. Then F^n) = /(tt) — [/(t)] is a vector held of A^. 

The differential equation is Stt = F{ 7 i), see Ay and Erb 2005| , [Pistone [2010 , 2013 . 


B.l. Simplex parametrizations. Let A 2 be the simplex as a sub-variety of 
A2 = {tt = ( 770, 77i, 77o) e I 77 o 77i -h 772 = 1} 

We consider different parametrizations. 


B.1.1. Solid simplex. 


770 = 1 - 6^1 - 02 , 

771 = 01, 

712 = O 2 , 


01 = 77i, 

6*2 = 772, ’ 


{(01,02) +02 < 1} 
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The Jacobian of 0 h-)■ 7r(0)) is 


-1 -1 

(26) J(6>) = 1 0 

0 1 

B.1.2. Exponential family. The exponential family 

(27) 


^ ^ g0iXi+e2X2-v>(0) 


i 

^1 

^2 

0 

0 

0 

1 

1 

0 

2 

0 

1 


-0(0) = log (l + e^^ + e*^^) 


gives 

(28) 


TTo = (1 + e®i + 

TTi = e^i(l + e^i + e®2)“^, , 

712 = e^2^X -p g^*! -j_ 


O 2 = log^, ’ 


-0' {(01,02) eM^}. 


As ni{ 6 ) = -^ 7 p{ 6 ), i = 1,2, then (711,712) (the solid simplex parameters) are the 
expectation parameters of the exponential family (27). 

The Jacobian is 



_e^'i 

_e^'2 


—TloTli 

—71o7l2 

(29) J(0) = (1 + e^i-7e®2)-2 

e0i(i + e®2) 

_ Qdl+92 

= 

711(1 - 71i) 

— 71i7l2 


_0^1+^2 

e®2(i + gei) 


—71i7l2 

to 

1 

to 


Notice that, if Z{ 6 ) = then 
(30) J(0) = 

B.1.3. Projective parametrization. 

Tio = (l-0l-02)■^ 

Til = 0i(l— 01 — 02) , 

712 = 02(1 - 01 - 02 )"^ 

The Jacobian is 


■VZ(0)-1' 


VZ(0)-i 

Hess pi (0) 


Var^ (Xi,X2)_ 


01 = 

02 = ’ 

TTO ’ 






■ -1 -1 ■ 


■ -^0^ 

-^0 

(31) 

1 

1 

to 

1 

tc 

1 — 02 —01 

= 

7I'o(^0 ~ 7I2) 

— TIqTIi 



1 

to 

1 


—710712 

'^oi'^0 — TTl) 


B.2. Differential equations in different parametrizations. A differential eqnation 
^7r(t) = Fiirit)) in the simplex A2, considered as snb-variety of M^, compnted compo¬ 
nentwise, has the form: 

(32) 


Tio (t) = Fo (tIo (f ), Til (f ), 712 (t) ) , 

Til (t) = Fi (710 (i), 7 ii (i), 712 (i)), 

7 i 2 (i) = F2 (710 (i), 7 ii (i), 712 (i)), 


where F has to be orthogonal to the constant vector, i.e. T" • 1 = 0, or Fq -|- Fi -|- F2 = 0. 
Becanse of this condition, it is enongh to consider 


( 33 ) 


7ii(i) = Fi (1 - 7ri(i) - 7r2(i),7ri(i),7r2(i)) , 
7i2(i) = F 2 (1 - 7ri(i) - 7r2(i),7ri(i),7r2(i)). 
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The differential equation (32), written in the parameter 6 has the form ^7z{6(t)) 


F(7r(0(t))), i.e.: 

(34) = Mm)m = F(Mem 


B.2.1. Solid simplex. In this case, the Jacobian is (26) and 
We have: 

^7r(e(t)) = (-1.1.0)' ^^’^(9(1)) = (-1,0,1)' 


and Eq. (34) in matrix form become: 


■-1 

-f 


01' 

p2_ 


Eo' 

1 

0 


= 

El 

0 

1 



E 2 


which implies: 9i = Fi and 6*2 = -F2, i.e. Eq. (33). 


The choice to leave out Fq is of course arbitrary. 


B.2.2. Exponential family. In this case the Jacobian is (29), hence Eq. (34) becomes 

(1 + e®i +e®2)“^ 


_e^'i 


-e®2 


e®i(l + e®2) —e 


61+62 


361+62 g62(']^_^g6i 

We need to compute the inverse of the lower block, i.e. 



0i' 

k 


Eo' 


= 

El 



E2 


Hess'0(0) ^ = (1 + e®^ + e*^^) 

1 + e®i + e®2 


e^i (1 -f-e®2^ 

_e6ie^'2 g’ 


_e^'ie®2 


61062 062 -|- 


-1 


361+62 


362 


,361+62 


(1 + e®i) 

g 6 i +62 g6i(x + e®2) 


= (1 + e^i Te*^^) 


l+e^i 

e®l 


l+e^2 

e®2 


The differential equation in the parameters 6 is 


01 = (1 + e^'i + e®2) ^^(0) + (i + g 6 i+g 62 ) 

02 = (l + e^i+e^^j ^^(0) + ( 1 + g 6 i+g 62 j 1 ^ p^^Q^^ 

or, written in tt: 

F2 


(35) 


01 = - + - El + E 

^ ' TTi TTO y ^ TTO 


02 = E 


TTO 


El + + ^ 

' 7r2 TTO 


B.2.3. Projective parametrization. Eq. (34) in matrix form becomes: 


-1 -1 

(1_0 i_ 02)-2 | i _02 

-02 1-01 



whose solution in 0i, 62 is: 

01 = (1 - 0i) (1 - 01 - 02) El + 01 (1 - 01 - 02) E 2 

02 = 02 (1 - 01 - 02) El + (1 - 01 - 02) (1 - 02) E 2 
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and written in tt: 


(61 = (1 - VTa) TTg ^ Fi + TTiTTo ^ F2 

\ 02 = TTsTTg'^ Fi + (1 - 7ri)7r(;'2 F 2 

Notice that, in each parametrization, the condition Fq + Fi + F 2 = 0 is verihed. 


Appendix C. Replicator equations 


The differential eqnation of the replicator, see Ay and Erb 2005 


is: 


A = Xi {fi{x) - X- f) 

On the simplex we have: 

f ^i(7r) = vri(/i(7r)-7r-/) 

\ ^2(7r) = 712 (/2(7r) - 77 • /) 

where the expected valne of / is 

(37) 77 ■ / = /oTTo + /iTTi + f 2 'n '2 = fo + (/l — fo)'^! + (/2 — fo)Tl2, 
SO that 

(38) /^ - 77 ■ / = (1 - 7ri)(/i - fo) - 7T2{f2 - fo), 

(39) /2 - 77 ■ / = -7ri(/i - fo) + (1 - 7r2)(/2 - /o), 


and Eq. (36) becomes: 

(40) 

or 

(41) 


7^1 = TTi ((/i - /o)(l - TTi) - (/2 - f 0 ) 712 ) , 
F 2 = 712 (-(/l - /o)7ri + (/2 - /o)(l - 772 )) , 


'Ff 

1 

-1 




7ri(l-7ri) -771 772 /l “ /o 

— 77i772 772(1 — 772) /2 — /o 

Replacing Fi and F 2 in the differential eqnations with the different parametrizations, 
written as a fnnction of tt, we have, in the exponential parametrization. 


(42) 


- 4- ' 

— T — 


1 

77o 


_j_ 

TTO 


77i(l-7ri) -77 1 772 

— 77 i 772 772(1 — 772) 

0 1\ [771(1 - 77i) 


1 1 
1 1 


+ 


V7i 


0 


77n 


-1 


—77i772 
—77 i 772 772(1 — 772) 


77l 772 

77l 772 


+ 


1 - 77i -772 

-77i 1-772 


= h 


SO that the differential eqnations are 

(43) 

In the projective parametrization 

(44) 


ei = h- fo 
02 = /2 — fo 


(1 - 772)770 2 

-2 1 
TTiTTo 

77i(l-77i) -77 1 772 

772 77o 

(1 - 77i)77o 

—77i 772 772(1 — 772) 


1 


77, 


0 L 


1 - 772 77i 

772 1 - 771 


1 - 771 -77l 

—772 1 — 772 



77l 

o' 


"TTl 

TTO 

o' 


0 

772_ 


0 

7r2 

TTO _ 


hence the differential eqnations are 
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( 45 ) 


ei = 9 ,{ h - fo ) 

^2 = 6*2 (/2 — /o) 


Appendix D. Second order calculus 
The expectation parameters are 

^0i 

Vii^) = 


and the information matrix is 

1(6) = diag (r/(0)) - r]{e)r]{e)' 

The precision matrix is 


I ^ = diag (77) ^ + (1-|77|) ^ 11 '. 


In fact 


(diag (r/) - r/r/') (diag (r/) ^ + (1 - |r/|) Al') = 

/ + (1 — \r]\)~^ diag (r/) 11' — 7777' diag (77)”^ — (1 — = 

/ + (1 — |77|)“^77l' — 77I' — I77I (1 — |77|)“^77l' = I. 

The derivative in the direction h, of 77 ha diag (77) — 7 ) 7 ]' at 77 is 

h HA diag (h) — Ht]' — 7 ]h', 

hence, composing with 

di 7 ]{e) = I{ 6 )ei = (diag (77(0)) - 77(0)77(0)') = rii{e) (e^ - 77(0)), 

we obtain 

dil{e) = diag(77i(0) (e^ - 77(0))) - r]i{e) (e* - 77(0)) 77(0)' - 7 ]r]i{e) (e^ - 77(0))' 
= rii{ 0 ) (diag ((e^ - 77(0))) - (e* - 77(0)) 77(0)' - 77(0) (e^ - 77(0))') . 

The following equality is to be used below. 

I-^dJ = 

(diag (77)"^ + (1 - |77|)"^11') {rji (diag ((e* - 77)) - (e* - 77) 77' - 77 (e* - 77)')) = 
T]i diag (77)“^ (diag ((e* - 77)) - (e* - 7 ]) 7 ]' - 7 ] (e* - 77)') + 

?7i(l - | 77 |)”Ai' (diag ((e^ - 77)) - (e^ - 7 ]) 7 ]' - 7 ] (e* - 77)') = 


We conclude by briefly reviewing the computation of the metric connection (Levi-Civita 
connection) which is required by e.g., the computation of the Riemannian Hessian. 
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Let 7z(t) be an univariate statistical model. Let F{7z(t)) and G{7z(t)) be centered piv¬ 
otal quantities (vector fields of the statistical model). The variation of Cov(F(7r(f)), G{7z(t))) 
in the time is: 

^ (Cov(F(7r(t)),G'(7r(t)))) = ^(F(7r(t)), G(7r(t)))^(q = 

^^Fi(7r(t))Gi(7r(t))4(7r(t)) = 

E G.I ,+E (|g.) 4+E p-o. (I 4 ) = 

{|4G>„„ + (F, |g),(„ + (/)F,G>rt, + (/)G>„„ = 

The last line can be properly written as: 


'dt 


see 


do Carmo 1992 , Pistone 2013 


(| (F,(Mt)))U + (^("(‘))) G>.(.)+ 


{F(7r(()), I (G.(7r(()))"., + (/(^(*))) G)..,,,. 


which defines an operator (affine connection jdo Carmo 1992] or metric derivative Lang 

T^ ) 


(F,a) ^ VFck + ^/-^(V/ck)F 

If the model is such that /“^(7r(t))^ (J(7r(t))) = 0, then 


^ (Cov(F(7r(t)), G'(7r(t)))) = (T)(7r(t)))(L^, G')„(t) + (F(7r(t)), ^ (G'i(7r(t)))(Li)7r(t)- 
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